Modern large language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering. Transformers are central to this capability because they can encode information during training and retrieve it at inference. Existing theoretical analyses typically study transformers under idealized assumptions such as infinite data or orthogonal embeddings. In realistic settings, however, models are trained on finite datasets with non-orthogonal (random) embeddings. We address this gap by analyzing a single-layer transformer with random embeddings trained with (empirical) gradient descent on a simple token-retrieval task, where the model must identify an informative token within a length-$L$ sequence and learn a one-to-one mapping from tokens to labels. Our analysis tracks the ``early phase'' of gradient descent and yields explicit formulas for the model's storage capacity -- revealing a multiplicative dependence between sample size $N$, embedding dimension $d$, and sequence length $L$. We validate these scalings numerically and further complement them with a lower bound for the underlying statistical problem, demonstrating that this multiplicative scaling is intrinsic under non-orthogonal embeddings.

Could Ukraine hold a presidential election right now? Will Europe use frozen Russian assets to fund war? How can Ukraine rebuild China ties? 'Ukraine is running out of men, money and time' Ukraine's capital Kyiv came under Russian missile attack early on Thursday morning, the country's military administration said, with witnesses reporting the sound of explosions. There were no initial reports of casualties and the extent of damage from the attack was not known.

Twelve miners have been killed by a Russian drone strike in eastern Ukraine, the country's largest private energy firm has said. DTEK said a bus carrying workers after a shift in the Dnipropetrovsk region had been targeted in Sunday's attack. At least seven people were injured. Earlier, at least two others were killed and nine injured in separate Russian attacks overnight and on Sunday. The victims included six people hurt when a drone hit a maternity hospital in Zaporizhzhia.

Could Ukraine hold a presidential election right now? Will Europe use frozen Russian assets to fund war? How can Ukraine rebuild China ties? 'Ukraine is running out of men, money and time' 'Trauma does not define us': Living with loss in wartime Ukraine She pauses, looking up at the photograph fixed to the gravestone. His face bears a striking resemblance to hers.

Ukrainians brace for -20C despite energy truce: 'It will be a catastrophe' It's not the electricity cuts that Yulia Hailunas struggles with most after the Russian airstrikes. Like so many Ukrainians, she's had no central heating since Russia launched a wave of targeted attacks on the power grid in January. So Yulia now lives in a long, quilted coat and hat in her flat, and rests her feet on a saucepan-full of hot water to keep them from freezing. If that's not enough, she lifts weights for 10 minutes to get warm. When the weather outside is above zero, it's just about bearable.

Could Ukraine hold a presidential election right now? Will Europe use frozen Russian assets to fund war? How can Ukraine rebuild China ties? 'Ukraine is running out of men, money and time' Ukrainian President Volodymyr Zelenskyy said that a state of emergency was being declared for Ukraine's energy sector, after repeated Russian attacks destroyed electricity and heat infrastructure. Zelenskyy said he asked the government to review curfew restrictions during "this extremely cold weather".

Could Ukraine hold a presidential election right now? Will Europe use frozen Russian assets to fund war? How can Ukraine rebuild China ties? 'Ukraine is running out of men, money and time' Russian drone attack kills 4 in Ukraine's Kharkiv as peace remains elusive A Russian drone attack on Ukraine's northeastern city of Kharkiv has killed at least four people and wounded six, officials have said, just hours after Washington accused Moscow of "dangerous and inexplicable escalation" of the war and as a peace deal remains distant. Kharkiv Regional Governor Oleh Syniehubov said Tuesday that the death toll from the attack on the outskirts of the frequently targeted city, just 30km (19 miles) from the border, had risen to four.

Romance and parenthood feel remote in Ukraine: 'I haven't had a date since before the war' Sitting in a wine bar in Kyiv on a Saturday night, Daria, 34, opens a dating app, scrolls, then puts her phone away. After spending more than a decade in committed relationships she's been single for a long time. I haven't had a proper date since before the war, she says. Four years of war have forced Ukrainians to rethink nearly every aspect of daily life. Increasingly that includes decisions about relationships and parenthood - and these choices are, in turn, shaping the future of a country in which both marriage and birth rates are falling.

Russia used its hypersonic Oreshnik missile in an attack on Ukrainian infrastructure. Ukrainian President Volodymyr Zelenskyy called for an international response.

Service members take part in what the Russian Defense Ministry said was the deployment of a nuclear-capable hypersonic Oreshnik missile system in Belarus, in a still image taken from a video released on Dec. 30. Russia's military says it has fired its hypersonic Oreshnik missile at a target in Ukraine in response to what it described as an attempted Ukrainian drone strike on one of Russian President Vladimir Putin's residences, something Kyiv has called a lie. It is the second time that Russia has used the intermediate-range Oreshnik, a missile that Putin has boasted is impossible to intercept because of its reported velocity of more than 10 times the speed of sound. The missile is capable of carrying nuclear warheads as well as conventional ones, but there was no suggestion that the one used in the overnight attack had been fitted with anything other than a conventional warhead. The Russian Defense Ministry said the strike had targeted critical infrastructure in Ukraine. It said Russia had also used attack drones and high-precision long-range land and sea-based weapons.